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Executive Summary 



The After-School Corporation (TASC) has completed its sixth year of 
operations, working with public and private partners across New York City and 
the state to develop and operate school-based after-school services for public- 
school students in the elementary and secondary grades. Since its founding in 
1998, TASC has emphasized twin goals of increasing the availability of after- 
school opportunities and enhancing the quality of after-school services. 

To assess TASC’s effectiveness, four foundations including the Charles 
Stewart Mott Foundation, the Carnegie Corporation of New York, the William T. 
Grant Foundation, and the Atlantic Philanthropies supported an external 
evaluation that answered questions about: quality and scale in program 
implementation, program effects on participating students, and program practices 
linked to student success. The evaluation collected data over four school years 
from 96 TASC after-school projects and their host schools in New York City. Its 
student sample numbered 52,000 after-school participants and 91,000 students 
who were enrolled in TASC host schools but not participating in TASC projects. 



Are TASC Services Meeting High Expectations for 
Quality? 

In 1998, TASC’s developers designed a program model intended to 
demonstrate quality by: recruiting students likely to benefit from after-school 
learning experiences, promoting high levels of after-school enrollment and 
attendance, employing well-qualified staff, building strong relationships with host 
schools and parents, delivering activities that promote learning growth and expose 
students to positive new experiences, providing training and technical assistance 
to help staff improve their services, and encouraging fiscal independence. 

Over the evaluation period, TASC programs became increasingly 
systematic in their operations and took many steps to improve their effectiveness. 
Examined altogether, these efforts provide evidence of program quality. 

■ TASC projects recruited large numbers of students 
(approximately 50,000 students in 2003-04) from schools that 
enrolled high percentages of students at risk of educational failure 
due to poverty, low achievement, and other factors. In 2001-02, 
site-level after-school enrollments averaged 289 students. 

■ TASC projects retained these enrolled students and encouraged 
high levels of after-school attendance, with a median attendance 
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rate of 85 percent among TASC participants in grades PreK-8. 
Projects strongly encouraged daily attendance. Among PreK-8 
students who participated one year (and could attend a second year 
because they were in a school that hosted a TASC project), 63 
percent also participated a second year. 

Site-level coordinators were well qualified for their jobs, with 86 
percent holding a bachelor’s degree and 40 percent holding a 
master’s degree. Eighty-five percent of coordinators had at least 
three years of work experience in social services, youth services, or 
education; 50 percent had held relevant management positions for 
three years or more; 21 percent were certified teachers. Other 
program staff (excluding site coordinators) were also qualified for 
their roles, with 80 percent reporting prior experience in social 
services, youth services, community organizations, or schools. 
Twenty-eight percent of after-school staff had a bachelor’s degree, 
and another 29 percent were college students. 

TASC projects played increasingly important roles within their 
schools, with 97 percent of principals reporting a strong 
relationship between the school and after-school program and 86 
percent reporting alignment or coordination between the school 
and after-school program. 

Program activities were most often designed to huild student 
skills in academic and cognitive areas, the arts, and social and 
cultural awareness. In the academic/cognitive area, the most 
common activities involved homework assistance, organized 
reading (students reading together), recreational reading (students 
independently reading materials they selected by themselves), 
math games, word games, and organized writing. 

TASC projects differentiated after-school activities from the 
regular school curriculum through the adoption of appealing 
activities that: were intended to result in a performance (97 
percent of programs) or written product (82 percent), involved 
groups of students working together on an extended 
interdisciplinary project (77 percent), or brought together multiple 
learning experiences centered around a common theme (76 
percent). 

TASC and other training providers whom it hired provided 
training and technical assistance to all after-school coordinators 
and to 78 percent of after-school staff. In general, this training was 
intended to help inform and build skills among training 
participants and to engender a sense of professionalism. Site 




coordinators particularly valued training that focused on the nuts 
and bolts of projeet management and operations, while other 
projeet staff most often valued training on how to engage and teaeh 
after-sehool partieipants. Over half of training partieipants (57 
pereent of eoordinators and 53 pereent of other staff) said that they 
had implemented ideas and strategies learned in training and that 
this information had improved their projeet operations. 

■ Consistent with TASC guidelines, after-sehool projeets gradually 

increased the portions of their project budgets that they 
supported through sources other than TASC. Projects reported 
an average of 37 percent of their budgets would be funded by non- 
TASC sources in 2001-02. 



Do Students Benefit from Participating in TASC 
Projects? 

School principals reported significant benefits for students who 
participated in TASC projects. Ninety-five percent said that TASC gave students 
access to activities not available during the regular school day, 79 percent said 
that participants’ parents expressed more positive feelings about the school 
because of the program, and 66 percent said that after-school participants received 
special opportunities to hone literacy skills. 

Analyses of data on academic performance and school attendance show 
that participation in TASC activities was linked to improvements in both areas, 
especially for students who participated regularly in TASC programming over 
two consecutive years. For largely technical reasons, benefits were clearer and 
more readily measurable at the elementary and middle grades than in high school. 

At the elementary- and middle-grades level: 

■ Analyses indicate after-school benefits in mathematics 
achievement, especially for those participants who attended TASC 
projects regularly and for more than a year (effect sizes of 0.13 for 
one-year active-or consistent-participants and 0.79 for two-year 
active participants). Comparable findings in reading and English 
language arts were not apparent, although participants in some 
projects consistently outgained comparable nonparticipants in this 
area. 

■ Participants showed greater gains in school attendance than did 
nonparticipants, with the greatest benefits at grades 5-8. For 
example, the difference in the attendance gain of participants and 
nonparticipants measured from seventh to eighth grade was 1.5 




percentage points or the equivalent of a net gain of 2.7 school days 
in a 181-day school year (effect size of 0.17). 

At the high school level; 

■ After-school participants passed more Regents exams and 
earned more high school credits than did nonparticipants, but 
conclusions from this evidence are limited by the fact that high 
school participants were already achieving at substantially higher 
levels than nonparticipants on average before entering a TASC 
project. 

■ The difference that reflects the clearest association with TASC 
high school participation is in improved school attendance. 

When compared to nonparticipants with similar records of prior 
school attendance, TASC participants showed significantly more 
positive school attendance after a year of TASC participation 
(although the attendance of both participants and nonparticipants 
declined over grades 9-12). 



What Practices Are Associated with the Greatest 
Benefits for Students? 

The evaluation examined the practices and characteristics of TASC 
projects with varying records of student academic growth to identify those 
practices and characteristics that effective projects held in common. This analysis 
necessarily excluded the program features shared by all TASC projects. In rank 
order of statistical importance, the following program characteristics were linked 
with student gains in both mathematics and reading/English language arts: 

■ High frequency and duration of activities focusing on academics 
and cognitive development 

■ Employment of a project site coordinator who is licensed to 
teach 

■ High frequency and duration of activities focusing on fitness, 
sports, and recreation 

■ A requirement that after-school staff submit activity plans for 
advance review by the site coordinator 

■ A project staff in which at least one out of four staff members has a 
four-year college degree 
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Together, these characteristies suggest that effective programs were likely 
to be staffed and managed with a clear intent to promote academic learning, often 
through project-based, interdisciplinary activities that engaged students in 
learning experiences that differed from those of the regular school day, as 
indicated in survey data. Effective projects also provided opportunities for 
exercise and fun after school. Information on these patterns can help after-school 
intermediaries like TASC decide whether to adjust their program models or their 
assistance and guidance to programs and whether to monitor certain elements of 
program operation with special care. 
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BUILDING QUALITY, SCALE, AND EFFECTIVENESS 
IN AFTER-SCHOOL PROGRAMS 



Summary Report of the TASC Evaluation 



In September 2003, The After-Sehool Corporation (TASC) began its sixth 
year of operation in New York City, working with publie and private partners 
aeross the eity and the state to develop and support sehool-based serviees for 
public-sehool students in the elementary and seeondary grades. From its 
founding, TASC has worked to advanee two interrelated goals: 

1 . Inerease the availability of after-sehool opportunities by providing 
resources and approaches for establishing new projects and 
expanding existing projects 

2. Enhance the quality of after-school projects by incorporating 
research-based components that are associated with student 
success and program sustainability 

These goals have led TASC to engage in rapid development and scaling up of 
after-school programming, in itself a major logistical accomplishment, and also 
the implementation of project monitoring, technical assistance, and professional 
development, along with the creation of special initiatives to address major needs 
and create new partnerships to enhance TASC’s network of after-school projects. 

Underlying TASC’s goals is its central 
mission, which is to promote high-quality 
after-school programming as an appropriate 
public-sector responsibility. Historically, 
after-school programming has been seen as a 
private good, supported by private citizens 
and from which private individuals, especially 
children and their families, reap benefits. 

Although this view is not universal, a sense of 
public responsibility for after-school 
programs has generally characterized only 
those communities with high levels of both 
resources and public commitment to 
supporting children and families. And in 

these few communities, after-school programs are often the first public 
expenditures to be cut when budgets tighten. 



Profile of TASC Projects 

In school year 2003-04, TASC 
supported a total of 242 school- 
based after-school projects, 
Including 186 projects In New York 
City and 56 projects located 
elsewhere In the state. TASC- 
afflllated projects In 2003-04 
served approximately 50,000 
students, 41 ,200 of whom were 
enrolled In the public schools of 
New York City. 
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TASC’s mission, in effect, calls for it to demonstrate that high-quality 
after-school programs can be created, operated, and sustained in partnership with 
public schools and with other public and private partners. A central proposition 
of this mission is that after-school programs can attract significant numbers of 
children on a regular basis and can offer these children important developmental 
opportunities, all at no out-of-pocket cost to participants or their families. Finally, 
according to this mission, these programs and the opportunities they offer can 
increase the likelihood that participants will succeed in school and in life 
generally. 

An important element in achieving TASC’s mission is the dissemination 
and use of evidence produced and reported by evaluation. Through objective, 
third-party evaluation, TASC expects to demonstrate that it can create, operate, 
and sustain partnerships that support after-school services for significant numbers 
of children. To help achieve its mission, TASC has used evaluation findings to 
determine whether the program’s after-school services can offer learning 
opportunities that benefit participants in measurable, significant ways. 

Under the TASC approach, after-school services are provided through a 
partnership between a public school (known as the host school) and a local 
nonprofit organization with ties to the community served by the school. All 
students enrolled in the host school are eligible to participate in the after-school 
project, which provides services from the end of each school day to 
approximately 6 p.m. Services are free of charge and intended to supplement the 
learning experiences of the regular school day. Programming generally 
emphasizes academic enrichment, homework assistance, the arts, and recreation. 
The intent of this program approach is to combine community connections, child 
and youth expertise, cultural resources, and specialized foci of selected nonprofit 
organizations with the academic focus, facilities, and access to students provided 
by public schools. 

The budget for the TASC initiative in the 2003-04 school year was $97.5 
million, up from $14 million in Year 1, $36 million in Year 2, $61 million in Year 
3, $76.8 million in Year 4, and $87.9 million in Year 5. The current budget mix 
reflects a growing concentration of resources from public sources. Approximately 
75 projects receive funding as part of partnerships with New York State under its 
Advantage After-School Program and 60 projects participate in partnerships 
supported by the federal 2U* Century Community Learning Centers Program, 
currently administered in New York by the State Education Department. Another 
five projects receive 2U* Century funds directly as a result of proposals written by 
TASC. In addition, 39 TASC projects receive funding from the federal 
government’s AmeriCorps program, 28 receive support through TASC’s 
partnership with New York City Department of Youth and Community 
Development’s Beacon program, and several other after-school projects benefit 
from grants received by TASC from New York State’s Extended Day Violence 
Prevention Program and the city’s Workforce Investment Act program. 
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This evaluation sought to answer three central questions: 



1 . Are TASC services meeting high expectations for quality? 

2. Do students benefit from participation in TASC projects? 

3. What practices are associated with the greatest benefits for 
students? 

To address each question, the evaluation focused primarily at the level of 
the overall initiative, with the units of measurement focused at the student level 
for the second question, and at the project level for the first and third questions. 
Practically speaking, this meant that the evaluation examined the experiences of 
individual students and projects to reflect on the experiences of the initiative as a 
whole, not to report on projects individually. 

The evaluation is grounded in the program’s theory of change, as first 
described in Reisner, White, Birmingham, and Welsh (2001) and quoted below: ^ 

1. The supply of affordable, high-quality after-school programs open 
to all children and youth is not adequate to meet the demand. 

2. The private sector and parents lack the resources and institutional 
access required to equitably meet the need for after-school 
services. 

3. Public sector investment can play a central role in expanding and 
improving after-school opportunities for children and youth. 

4. An intermediary organization, such as TASC, is necessary to 
promote, cultivate, coordinate, and support local partnerships that 
provide high-quality after-school programs. Strategies that the 
intermediary uses to provide support need to include training, 
technical assistance, and volunteer recruitment. These supports 
will promote program quality, including continuity in staffing. 

5. By developing strong local partnerships, TASC will identify and 
increase a wide range of public and private supports that can be 
effectively targeted to expand and improve after-school services, 
including training and technical assistance. 



* Reisner, E.R., White, R.N., Birmingham, J., & Welsh, M. (2001). Building quality and 
supporting expansion of after-school projects: Evaluation results from the TASC after-school 
program’s second year. Washington, DC: Poliey Studies Associates. 
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6. Using financial resources drawn from many sources and a 
program design based on core elements that promote good 
practice in both education and youth development fields, TASC 
will award grants to nonprofit organizations to operate after- 
school projects whose designs are research-based and 
comprehensive in scope. 

7. The TASC projects will extend after-school services to children 
and youth in their host schools, using the resources provided by 
TASC and others to support nonprofit sponsoring organizations. 
TASC grants will explicitly promote collaboration between schools 
and communities, with the after-school projects bridging these two 
cultures. 

8. In their first year of operations, projects will establish procedures, 
staffing, and schedules to permit the delivery of planned activities 
and services. In subsequent years, the projects will focus on 
refining the quality and effectiveness of their activities and 
services, as they enlarge their capacity to serve more students. 

9. As a result of effective program designs that address the needs of 
working parents and their school-age children, enrolled students 
will attend the after-school projects regularly as part of an 
extended learning platform supported by their schools. 

10. By participating regularly in after-school activities that are 
distinctive from but connected to the school day, students will 
improve their academic skills and knowledge, gain new 
experiences, take advantage of opportunities for recreation and 
artistic expression, and develop psychosocial skills that promote 
positive youth development. 

11. The increased availability of high-quality after-school services 
will: (a) provide participants with opportunities for educational 
enrichment and socialization; (b) increase parents’ peace of mind 
regarding their children’s safety; and (c) improve parents’ ability 
to succeed in employment and/or their own education. 

12. Through the documentation of after-school activities, TASC’s 
independent evaluator will collect evidence describing the delivery 
and effectiveness of after-school services from participating 
schools, parents, and students. 

13. The evidence, in combination with organized, broad-based 
advocacy, will persuade the public and policymakers that after- 
school programs are necessary, feasible, and affordable. 
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14. Support for the programs will become widespread, and public 

officials will increasingly invest in universally available and 
sustainable programs as a public-sector responsibility. 

The change theory has framed the evaluation in two respects. First, it 
pointed evaluators toward categories of data that were particularly important to 
collect in order to tell the story of TASC’s operations and effects. Second, it 
guided data analysis, both in terms of highlighting the types of data that were 
most critical to understanding the initiative and also in helping evaluators to trace 
backwards from actual participant effects to program factors that may have played 
a role in producing those effects. 

Data collection for virtually all evaluation purposes focused on the group 
of TASC projects in New York City that were first funded in the first two years of 
TASC’s operations, 1998-99 (Year 1) and 1999-2000 (Year 2). This set of 
projects included all 50 projects first funded in Year 1 and all 50 projects first 
funded in Year 2. Ninety-six of these projects continued operation in Year 3, and 
95 were operating in Year 4. These projects offered the longest period during 
which evaluators could observe change in program implementation. Similarly, at 
the student level, these projects offered the longest possible period to observe 
TASC participation patterns and associated changes in educational performance. 

Evaluators collected data through surveys, site visits, and the review of 
administrative records. The evaluation’s survey data collection is profiled below. 
Survey data were collected from projects first funded in Year 1 and Year 2, 
except that student survey data were only collected from projects first funded in 
Year 1 . Because the full sample size was not attained until Year 2 for the site 
coordinator, staff, and principal surveys, most cross-time comparisons reviewed 
in the report span Years 2 through 4. 



Profile of Survey Data Collected in the Evaluation, 
by the Year in Which Data Were Collected 



Respondent 

Group 


Yean 

(1998-99) 


Year 2 
(1999-2000) 


Years 

(2000-01) 


Year 4 
(2001-02) 


TASC site 
coordinators 


47 (94%)* 


94 (94%) 


95 (99%) 


79 (83%) 


TASC after- 
school staff 


906 (86%) 


1,155 (87%) 


1,571 (92%) 


1 ,369 (93%) 


School principals 


34 (68%) 


66 (66%) 


69 (72%) 


65 (68%) 


Students 


1,584 (82%) 


1 ,865 (87%) 


2,103 (88%) 


2,031 (98%) 


Parents 


1 ,292 (70%) 


1,630 (91%) 


0 


0 



* Information in parentheses indicates response rates for surveys of TASC site coordinators and 
school principals. For TASC staff, students, and parents, figures in parentheses indicate the 
percent of TASC projects providing one or more survey responses. 
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In each year of the study, the evaluation collected data through interviews, 
observations, and focus groups conducted in visits to a sample of TASC projects. 
Site-visit samples included 10 projects in Year 1, 15 projects in Year 2, 15 
projects in Year 3, 14 projects in Year 4, and 14 projects in Year 5. In general, 
the evaluation used site-visit data to explore the relationships, issues, and tensions 
surrounding the design and implementation of TASC projects. Examples cited 
throughout this report are drawn from these visits. 

In 1998, TASC and PSA developed a student tracking system to register 
the following: the date each student enrolled in TASC activities, the student’s 
withdrawal date, and the student’s daily after-school attendance during the period 
between enrollment and withdrawal. This system produced data on patterns of 
students’ after-school enrollment and attendance. In addition, by using the 
identifying information supplied by the system, the evaluation cross-referenced 
TASC participants with the New York City Department of Education’s student 
data. This effort yielded information on participants’ biographical characteristics 
(date of birth, sex, race/ethnicity, eligibility for free or reduced price lunch, and 
whether the student was a recent immigrant), their eligibility for special 
instructional programs serving English Eanguage Eearners and students with 
disabilities, and their educational characteristics (grade in school, years of 
enrollment in a particular school, annual school attendance rates, and achievement 
on state and citywide tests of reading and math). Data on additional educational 
characteristics were obtained for high school students, including students’ 
accumulation of high school credits and Regents test performance in English and 
math. In addition, by knowing the TASC participants in each school, evaluators 
were able to identify students who did not participate in TASC and to obtain the 
same information on these nonparticipants as it obtained on participants. (The 
evaluation defined a nonparticipant as any student who attended a TASC host 
school while a project was open and who did not participate in a TASC project in 
any year.) The final student sample for the evaluation contained information on 
52,355 students who participated in TASC programming and on 90,806 students 
in the same schools who did not participate in TASC services. 



Are TASC Services Meeting High Expectations for 
Quality? 

Evidence gathered in the evaluation indicates that the TASC program 
met expectations for quality in key areas, as local projects became more fully 
integrated into their host schools and as program operations became 
increasingly systematized over the evaluation period. TASC’s program model, 
its partnerships throughout New York City and state, and its provision of training 
and technical assistance are all intended to support high levels of after-school 
program quality. As outlined in the program’s theory of change, TASC believes 
that high quality is essential to attract and retain students in after-school programs 
and also to promote concrete benefits for students, including improvement in their 
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academic skills and knowledge, their exposure to new experiences, their 
enjoyment of recreation and artistic expression, and their development of 
psychosocial skills capable of promoting positive youth development. 
Accordingly, the evaluation has measured program quality through several 
distinct lenses, including the recruitment of students likely to benefit from after- 
school learning experiences, promotion of high levels of after-school attendance, 
development of strong relationships with host schools and parents, delivery of 
activities that promote learning growth and that expose students to positive new 
experiences, provision of training and technical assistance to help staff make their 
programs better, and encouragement of fiscal independence. 



Student After-School Participants and Their Schoois 

TASC projects served large numbers of students during the 
evaluation period-approximately 50,000 students in 2003-04, with students 
drawn mainly from public schools that enrolled high percentages of students 
at risk of educational failure due to poverty, low achievement, and related 
factors. High levels of participation by these students are consistent with the 
program’s theory, which is premised on the provision of services to children of 
families who are not able to pay for high-quality after-school services out of their 
own pockets. In addition, information on the student subgroups that derive the 
most academic benefit from TASC participation, as discussed later in this report, 
suggests that students at educational risk are especially likely to experience 
educational gains from participation. 

Compared to New York City’s public schools overall, the schools in the 
evaluation sample enrolled higher percentages of students from low-income 
families, low-achieving students, and black or Hispanic students. This difference 
was most pronounced for schools hosting PreK-8 projects and less pronounced for 
schools hosting high school projects. 

Within PreK-8 host schools, about 32 percent of students in each school 
enrolled in the TASC project. At grades 9-12, 59 percent of students in each 
school enrolled, although that figure is misleading because several high schools 
automatically enrolled all students in the TASC program. 

The PreK-8 students who participated in TASC programs within 
their schools closely resembled the nonparticipating students in the same 
schools, on measures of family income, gender, receipt of special education, 
status as English Language Learners and as recent immigrants, and prior 
educational performance. Although racial/ethnic characteristics were very similar 
across participants and nonparticipants, black students were more highly 
represented among participants (37 percent) than among nonparticipants (28 
percent). 
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Overall, low-income students (as determined by eligibility for subsidized 
meals) constituted 91 percent of PreK-8 participants and 75 percent of high school 
participants-although this difference may be smaller than it appears, because high 
school students are generally reluctant to identify themselves as eligible for 
subsidized meals. Hispanic and black students accounted for 85 percent of 
participants at PreK-8 levels and 78 percent of grade 9-12 participants. 



Students’ Patterns of After-School Enrollment and Attendance 

Trends in TASC program enrollment and attendance suggest that TASC 
projects are serving large numbers of students on a relatively frequent basis 
and that both project-level enrollment and attendance have increased 
somewhat over time. These patterns are promising with regard to program 
appeal to students and their parents and also with regard to the likelihood of 
program effectiveness. Findings presented in interim reports of this evaluation 
pointed to the importance of consistent after-school attendance in ensuring the 
level of program exposure that permits participants to benefit from high-quality 
after-school experiences. 

Among the 75 projects serving grades PreK-8 for which adequate 
enrollment data were available for Years 2, 3, and 4, project enrollment remained 
relatively stable, increasing slowly across program years. A factor promoting 
enrollment stability was TASC’s policy of adjusting grant amounts during the 
program year to align the amounts with actual project attendance. In Year 2, the 
average enrollment was 236 students, increasing to 278 in Year 3, an increase of 
18 percent. In Year 4, the average enrollment in these projects increased slightly 
to 289, a 4 percent increase. From Year 2 to Year 4, project enrollment increased 
by an average of 22 percent. 

In addition, the proportion of students who participated in the after-school 
project for an entire school year increased moderately between Years 3 and 4. In 
Year 4, 77 percent of participating students continued to attend the project every 
month through May, compared with 74 percent of students the previous year. 
Among PreK-8 students who participated one year and could attend a second year 
because they continued in a school that hosted a TASC project, 63 percent also 
participated a second year. 

TASC’s project structure and operations were premised on PreK-8 
students attending the program regularly, preferably five days per week. To 
analyze students’ level of TASC participation, the evaluation first categorized 
each participating student as an active participant or a nonactive participant. In 
grades PreK-8, an active participant was one who attended a TASC project at 
least 60 days during the school year (out of the typical 160 days of project 
operations) and also attended at least 60 percent of the days that it was possible 
for the student to attend, or an average of three days per week. Because of the 
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importance of regular, frequent attendanee in the TASC program model, most of 
the analyses presented in this report eompare aetive partieipants to 
nonpartieipating students. 

In 2001-02, 71 pereent of PreK-8 participants in TASC projects met the 
eriteria established for active partieipation. Among TASC partieipants in grades 
PreK-8 during the 2001-02 sehool year, a project attendance rate of 60 pereent 
represented the 18**^ pereentile of attendanee rates among all TASC participants, 
and a project attendance rate of 80 pereent represented the 40* pereentile point. 
This means that 18 pereent of all TASC partieipants at this grade span had a 
TASC attendanee rate of 59 pereent or lower, and 40 pereent had an attendanee 
rate of 79 pereent or lower. 

Average TASC attendance rates for PreK-8 students increased in 
each of the four years of TASC program operation that were assessed, as seen 
in the distribution of student attendance rates. The midpoint on the attendance- 
rate eontinuum inereased from 78 pereent in Year 1 to 80 pereent in Year 2 to 83 
pereent in Year 3 and finally to 85 pereent in Year 4. 

The distribution of partieipating students based on the number of days 
attended during the sehool year showed a similar trend, leveling out in Years 3 
and 4. The median days attended in 1998-99 were 80 days, in 1999-2000 the 
median was 99 days, in 2000-01 it was 109 days, and in 2001-02 it was 107 days. 
This caleulation excludes attendance data from sites that submitted data for fewer 
than eight months in a sehool year and the sites that operated only a half-year in 
1998-99 or 1999-2000. 

For students in grades 9-12, TASC’s project structure and operation were 
premised on students attending the projeet on a less frequent basis. Aeeordingly, 
evaluators established a different threshold for eategorizing a student in grades 9- 
12 as an aetive partieipant: a minimum of 20 days over a school year and 20 
pereent of the days that it was possible for the student to attend, an average of one 
day per week. During 2001-02, 47 pereent of partieipants in grades 9-12 met the 
eriteria for active partieipation. Among TASC partieipants in grades 9-12 during 
the 2001-02 school year, a project attendance rate of 20 pereent represented the 
38* pereentile of attendance rates among all TASC participants, and a project 
attendance rate of 60 pereent represented the 90* pereentile point. Average 
TASC attendanee rates for students in grades 9-12 fluetuated slightly aeross the 
four years of TASC program operation. 



9 




Characteristics and Work Experience of Project Staff 

Evaluators’ analysis of project characteristics associated with learning 
gains for students underscore the importance of staff background and staff 
supervision in promoting service effectiveness for students, as discussed in a later 
section. The following discussion highlights the staff characteristics and 
experiences that evaluators found to be most closely related to student gains. 

Site coordinators’ background and job satisfaction. In general, TASC 
site coordinators were well educated, with 86 percent holding at least a 
bachelor’s degree and 40 percent holding at least a master’s degree. Although 
average education levels of site coordinators were high, they declined (but not 
significantly) over Years 2, 3, and 4. An explanation for the decline is that over 
time projects promoted assistant site coordinators into lead positions. These 
individuals had not needed the higher education levels to be hired as assistant 
coordinators, but they proved themselves to be effective in subordinate positions 
and were eventually promoted based on their work experience and performance. 



Average Educational Level of Site Coordinators, 
1999-2000, 2000-01, and 2001-02 



Educational Level 


Year 


Percent of Coordinators 


N 


Graduated from four-year college or 
higher 


Year 2 


94 


73 


Years 


90 


69 


Year 4 


86 


70 


Master’s degree or higher 


Years 


5S 


73 


Years 


45 


69 


Year 4 


40 


70 



Coordinators demonstrated high levels of relevant prior work 
experience. In Year 4, 85 percent of site coordinators reported at least three years 
of experience working in social services, youth services, community 
organizations, or educational organizations before working in a TASC after- 
school project; this level remained stable over time. In the same year, 50 percent 
of site coordinators reported at least three years’ experience in managing social 
services, youth services, community organizations, or educational organizations 
prior to working in a TASC after-school project; this level also remained fairly 
stable. Site coordinators in Year 4 were very likely to have at least three years of 
experience providing direct services to youth, with 78 percent reporting this level 
of direct youth-serving experience, a level that remained stable over the data 
collection period. 

About a fifth of site coordinators (21 percent) reported that they held 
some type of state teaching certification from New York or another state. Of 
these site coordinators, 20 percent held a regular license to teach in New York 
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City, 20 percent held a New York State certificate, and others held provisional 
certificates of various types, with some site coordinators holding more than one 
type of teaching certificate. 

Based on a scale constructed from items on the site coordinator survey, the 
evaluation rated site coordinators’ overall satisfaction with their jobs. The results 
showed that 95 percent of site coordinators reported a high level of job 
satisfaction. This level of job satisfaction has been a consistent feature of site 
coordinator reports since program inception. 

Project staff background and job satisfaction. As in previous years, staff 
(excluding site coordinators) reported relatively high levels of prior 
education, with 40 percent holding a two-year college degree or higher. Nineteen 
percent of responding staff had not yet graduated from high school. 



TASC Project Staff with Various Leveis of Education, 2001-02, 

in Percents (N=1,208) 



Education Level 


Percent of Staff 


Less than high school 


19 


High school or GED 


16 


Some college 


25 


Completed two-year college degree 


12 


Completed four-year college degree 


9 


Some graduate work 


5 


Master’s degree or higher 


14 



Over half of project staff (57 percent) who were not working as site 
coordinators were themselves enrolled as students while working in a TASC 
project. This pattern was not surprising. The part-time nature of many TASC 
jobs made after-school employment compatible with course enrollment. Most 
TASC staff who were enrolled in education programs were college students. 



Staff Enroiiment in Educationai Programs, 2001-02, in Percents 

(N=1,208) 



Education Enrollment Level 


Percent of Staff 


College 


29 


High School 


17 


Graduate program 


8 


Other educational programs 


3 


Not enrolled in educational programs 


43 
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In survey questions about their work experience, staff respondents 
reported extensive experience in relevant fields before becoming a member of 
the after-school staff, as shown below. 



Types of Work Experience Reported by Staff, 2001-02, in Percents 

(N=1,259) 



Work Experience 


Percent of Staff 


Social services, youth services, 
community organizations, or education 


80 


Work in a school 


67 


Three or more years in social services 
or education 


53 


Three or more years providing direct 
services to children 


43 


Teacher certification 


18 



Almost a third (32 percent) of staff reported that they worked in the host 
school in some capacity during the school day. Thirteen percent said that they 
worked as classroom teachers during the school day, 9 percent worked as 
classroom assistants, and 10 percent worked in other positions, such as 
instructional specialist, pupil support staff, and administrator. In general, after- 
school staff who also worked in the host school during the school day reported 
that their dual role was a positive experience, as seen below. 



Reactions to Duai Roie by After-Schooi Staff 
Who Worked in the Schooi during the Reguiar Schooi Day, 2001-02, 

in Percents (N=274) 



Staff Response 


Percent Who 
Strongly Agree 


My relationships with some students 
have improved because I get to work 
with them after school 


50 


I get a chance to try new activities and 
teaching strategies 


36 


I have changed some activities or 
teaching strategies in my regular 
classroom 


22 


I am worried about getting burned out 
on teaching 


7 


I have less time for my regular teaching 
responsibilities 


4 
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Interviews with after-sehool staff and site eoordinators revealed many 
unexpeeted benefits of hiring teaehers from the regular sehool day. Perhaps most 
benefieial was the faet that regular day teaehers brought insights into after-sehool 
partieipants’ aeademie and developmental needs, and gained insight into their 
eurrent and future students by meeting them in a more relaxed setting. Also, 
regular day teaehers were more likely to have aeeess to sehool resourees, either in 
their own elassrooms or when using a eolleague’s elassroom. In addition, regular 
day teaehers who worked in the after-sehool projeet often served as advoeates for 
it, eneouraging other teaehers to share spaee or refer students. 

The same interviews, however, revealed some negative eonsequenees of 
hiring regular day teaehers: 

■ Projeets wanting to offer an experienee different from the regular 
sehool day sometimes had diffieulty shaping the tone of student- 
staff interaetions, partieularly when site eoordinators did not have 
full eontrol over whieh teaehers they eould hire. 

■ Some teaehers were exhausted by the end of the sehool day or 
found themselves short on planning time for the next day. At one 
sehool, the prineipal limited the number of hours a teaeher eould 
work at the after-sehool projeet. This poliey allowed more 
teaehers to earn extra ineome, while providing students with after- 
sehool staff familiar with eurrieulum that the regular sehool day 
and the after-sehool program shared. 

■ Certified teaehers were typieally more expensive to hire. 

■ Teaehers were unavailable during the regular sehool day for TASC 
training, and, eorreetly or not, teaehers pereeived their edueation 
and elassroom experienee as too advaneed for them to benefit from 
TASC training. 

Irrespeetive of personal and professional baekground, project staff 
reported high levels of job satisfaction, with 95 pereent of responding paid staff 
indieating that they were “highly satisfied” with their work in the after-sehool 
projeet. 



As in previous years, almost two-thirds (65 pereent) of projeet staff in 
Year 4 reported that they intended to return to their jobs in the upeoming year. 
Ten pereent said that they would not return, and 25 pereent said that they weren’t 
sure whether they would return. Among those not returning or not sure, the main 
reason was needing to work more hours or full-time. 

Staff supervision. Site eoordinators were asked whether their projeets 
held staff meetings at least onee a month, with 87 pereent reporting that staff 
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meetings were held at least that often. This represented an inerease in reported 
staff meetings from prior years. In addition, 71 pereent of site eoordinators 
reported that they held meetings of subgroups of staff at least onee a month. The 
typical staff-meeting agenda included discussion of new activities, upcoming staff 
development, policy changes (most often regarding student attendance), and the 
sharing of lesson plans. 

Eighty-two percent of site coordinators said that they required activity or 
lesson plans from at least some staff. Half (41 percent) of these site coordinators 
said that they regularly required activity plans from most or all staff. At some 
projects, the site coordinator kept activity plans on file, so that staff could borrow 
ideas from each other and ask the creator of a lesson plan for help in 
implementing it. 



Relationships with Host Schools 

Both after-school projects and their host schools placed a high value 
on creating and nurturing close ties between after-school and regular school 
day programs, or at least they did so in most instances. These efforts were 
visible in the engagement of principals and school staff with the after-school 
project staff, the level of coordination and integration between the after-school 
program and the regular school day, projects’ access to school space and facilities, 
and, ultimately, projects’ progress in building positive relationships with the host 
school. When ties between school and after-school created tight webs of support 
for children, these relationships were especially strong, and contributed 
significantly to program quality. 

In separate interviews during site visits, the principal, the site coordinator, 
and the executive director (or other senior representatives) of the sponsoring 
organization were asked about the collaboration between the host school and the 
after-school program, including how decisions about after-school programming 
and space were reached. A consistent picture emerged across the sites visited by 
evaluators. While the site coordinator communicated regularly with the principal, 
the coordinator had the leeway to hire and supervise staff as well as determine 
program activities and the weekly schedule. The principal typically acted as an 
after-school advocate (with the regular day staff) and a leader in logistical 
brainstorming around such issues as sharing space and wrapping around non- 
TASC after-school activities. The supervisor of the grantee organization helped 
launch the project, oversaw the budget, and responded to TASC compliance 
requirements (e.g., annual cost reports, quarterly narratives). 

In surveys in Year 4, virtually all principals of host schools (97 percent) 
reported a strong partnership between the school and the after-school 
project. This level of affirmation represented an increase from prior years, with 
87 and 90 percent of principals reporting a strong partnership in Years 2 and 3, 



14 




respectively. In addition, 86 percent of principals reported that the TASC project 
was aligned or coordinated with the school in some way. Principals reported a 
variety of means by which this alignment or coordination occurred, as shown 
below. While the use of all of these strategies increased between Years 2 and 4, 
according to principals, the integration of school themes and the identification of 
skill needs increased most during the period. 



Strategies for Augmenting the Reguiar Schooi Day, 
as Reported by Principais, 2001-02, in Percents (N=49) 



Strategies 


Percent 


The after-school program solicits input from 
the principal and teachers on skills in which 
students need help, and incorporates these 
topics into after-school activities 


86 


The after-school program adopts school 
themes for special projects 


65 


The after-school program uses school 
administrators (e.g., deans, assistant 
principals) to advise or monitor activities 


61 


After-school staff coordinate homework 
assistance with classroom teachers 


51 


The coordinator of the after-school program 
serves on a school planning team 


51 


Other 


10 



A factor that increased the coordination and integration of the after-school 
project with the regular school-day program was the employment of one or more 
of the school’s teachers in the after-school project. Principals were especially 
likely to note the positive effect of this cross-staffing on the alignment of 
curriculum and instruction. 

At a very tangible level, host schools demonstrated their acceptance of 
and engagement with the TASC after-school project hy sharing space and 
facilities. In general, after-school projects had good access to most school spaces 
and facilities in Year 4. Analysis of trends over time indicated that host schools 
increased the extent to which they shared many school spaces and facilities. One 
exception, however, was that they decreased the extent to which they shared their 
schools’ computers. Significant changes between Years 2 and 4 included the 
following: 

Increased access to: 

■ Playground/outdoor activity space, especially among elementary- 
grades projects 
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■ Telephone 

■ Auditorium, espeeially among elementary-grades projeets 

■ Storage spaee 

■ Fax 

■ Copier 

■ Gymnasium, among elementary-grades projeets only 

Decreased access to: 

■ Computer lab, although access rose for high school projects 

■ Classroom computers, although access rose for high school 

projects 

On several survey items assessing the relationship between the school and 
the after-school project, principals’ responses turned significantly more positive 
between Years 2 and 4 of the program. The following measures showed 
significant increases after Year 2: 

■ After-school staff reach out to teachers in the school to identify the 
needs of students 

■ After-school staff follow through with the commitments they make 
to the principal and other school staff 

■ After-school staff take care of the space the school provides the 
project 

■ Students are properly supervised by after-school staff 

Perhaps reflecting the growing popularity and demand for TASC services, 
the percent of principals’ positive responses decreased significantly on one 
measure: 



■ The after-school project has enough capacity to serve all interested 
students 

Principals noted several program elements that in their view needed 
attention, as shown below. 
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Principal Reports of Program Elements Needing Attention, 2001-02, 

in Percents (N=64) 



Program Elements 


Percent 


Qualifications of program staff other than the 
site coordinator 


56 


Quality of homework help 


52 


Quality of academic enrichment activities 


44 


Coordination/integration with the school 
curriculum 


41 


Coordination with the school 


31 


Number of staff 


22 


Staff turnover 


20 



Concerns about staff qualifications may be linked to many principals’ 
preference that the after-school project employ as many teachers from the regular 
school day as possible. While sponsoring organizations were keenly aware of 
principals’ hiring preferences, according to interviews with site coordinators, they 
still balanced these with the need to hire staff with an array of credentials and 
work experience. Their aim was to (1) keep staffing costs within budget, (2) offer 
a wide range of programming to students, and (3) meet goals of increasing 
opportunities for youth development during after-school hours. 



Relationships with Parents and Community 

In addition to building relationships with the host school, TASC projects 
were expected to reach out to parents and the broader community and to 
build strong relationships with these groups. During site visits, site 
coordinators reported that parent and community outreach presented challenges, 
however. By virtue of having to pick up their children at 6 p.m., parents of 
elementary school students were likely to be connected to the projects. 
Elementary school-based projects were also more likely than other projects to 
engage in after-school community service activities (e.g., park cleaning), 
sometimes also involving parents. This may be attributed to the fact that 
AmeriCorps members, who were required to lead a community involvement 
project, were almost exclusively placed in elementary school projects. 

Site coordinators reported in surveys that they sponsored various types of 
activities to involve parents and families in the after-school project, with the most 
frequently sponsored activities being recreational and cultural events. 
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Project Sponsorship of Activities for Parents and Famiiies, 
as Reported by Site Coordinators, 2001-02, in Percents (N=77) 



Activities 


Percent of 
Projects 


Opportunities to attend cultural or recreational 
events in the community 


72 


Parenting classes (e.g., classes to help parents 
learn about the school system and 
communicate with the school, how to help their 
children with schoolwork and prepare for tests, 
etc.) 


37 


Opportunities to hear from and talk with 
representatives from local agencies or other 
organizations (e.g., health, police, employment 
and training programs) 


36 


Classes to help parents develop their own skills 
(e.g., GED preparation or computer skills, etc.) 


24 


English as a Second Language classes 


18 


Other events and activities 


14 



Analyses demonstrate inereases in all of these activities across years. The 
increases in “opportunities to hear from and talk with representatives from local 
agencies or other organizations” and “opportunities to attend cultural and 
recreational events in the community” were significant. 

In most instances, projects engaged with the local community 
primarily through their sponsoring organization, which in every case was a 
private nonprofit organization. Most of these organizations were local 
community-based organizations whose main purpose was supporting or 
improving some important service or function (e.g., youth services, health care, 
housing, culture). Even when the sponsoring nonprofit was a cultural or other 
institution serving a citywide mission (e.g., a museum), it either had or desired a 
strong link to the community in which the after-school project was located. For 
this reason, TASC projects’ engagement with the community was grounded in 
their sponsoring organizations’ ongoing community outreach and engagement. 
According to site coordinators, people in the neighborhoods surrounding their 
projects were generally aware of them, with 86 percent reporting that the 
surrounding neighborhood was either very aware or somewhat aware of the 
project. High school projects were especially likely to engage with the 
community through development of internships for participants. 
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Training and Technicai Assistance 

Evaluation evidence suggests that projects’ participation in training and 
technical assistance was important in promoting project quality. Indeed, a 
distinctive feature of the TASC initiative is its commitment to training and 
technical assistance. Working through New York City-based training providers, 
TASC offered varied types of training to after-school project staff, with training 
opportunities tailored to specific subject areas and trainees. The evaluation 
examined these opportunities through data provided by TASC site coordinators 
and other staff When asked what types of training were most valuable, project 
staff identified topics that were most closely related to their core day-to-day 
responsibilities, especially how to involve and teach after-school participants, as 
being of greatest relevance to them. For site coordinators, these topics centered 
on the nuts and bolts of project management and operations. Other staff were 
most interested in learning how to design activities to engage, involve, and teach 
after-school students. 

All site coordinators participated in TASC training. Examined across 
three years of staff responses, the percent of other after-school staff who said they 
participated in after-school training as part of their job increased steadily, from 62 
percent to 68 percent to 78 percent in Years 2, 3, and 4, respectively. One reason 
for this increase was that sponsoring organizations developed and began to lead 
one- to two-week training and orientation sessions as part of the annual routine of 
many TASC projects. All of the sites visited in the evaluation opened their 
program year with a one- to two-week staff orientation that covered topics such as 
child-abuse detection, classroom management, and group cohesion-building, as 
well as specific instruction in leading arts and academic enrichment activities. 

Those staff members who did not participate in training were mainly staff 
who worked in the TASC project 10 or fewer hours a week (47 percent of all staff 
not participating in training), staff who were 21 years old or younger (17 percent), 
and staff who worked in the school during the regular day and also worked more 
than 10 hours a week in the after-school project (7 percent). 

An important measure of the effectiveness of training is the satisfaction 
expressed by training participants and their reports of using what they learned. 

The evaluation addressed questions on these topics to both site coordinators and 
other paid staff of the TASC projects. Site coordinators in Year 4 expressed a 
high degree of satisfaction with their training. This level of satisfaction reflected 
a steady upward shift, with site coordinators who reported that the services served 
their purposes completely increasing from 18 percent in Year 2 to 22 percent in 
Year 3 to 38 percent in Year 4. In interviews, site coordinators were equally 
positive about the training. At one project, the site coordinator commented that 
staff came back from sports training motivated to organize a Special Olympics for 
their students with special needs. Responses from other paid staff in Year 4 were 
consistent with feedback from site coordinators. 
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Both site coordinators and other paid staff responded positively, and 
similarly, to questions about whether they had implemented the ideas and 
strategies presented in the training and technical assistance. Most notably, 
over half of each group reported that they had implemented ideas and strategies 
from the training and technical assistance and that the implementation had 
improved the TASC project. Another third of each group was in the process of 
implementing ideas and strategies gleaned from the assistance. While responses 
from site coordinators became more positive after Year 2, staff responses 
remained about the same. 



After-School Project Resources and Expenditures 

The total TASC award in 2001-02 to 84 TASC projects for which 
expenditure information was available was $15,758,800. These grants for annual 
operating expenses were computed at $1,000 per student for projects that began 
operation in 1999-2000 and $980 for projects that began operation during 2000- 
01 . In addition, TASC paid sites $20 per student from the BOB allocation of $80 
per enrollee, bringing total awards to $1,020 and $1,000, respectively. TASC 
allocated resources to projects by borough and grade level, as follows: 



Expenditures by the TASC After-School Projects, 2001-02 (N=84) 



Project 

Category 


Total Amount 
Expended 


Number of 
Projects 


Average Expenditure 
per Project 


All Projects Providing Data 


1 All 


$ 


15,756,800 


84 




187,581 1 


By Borough 


Bronx 


$ 




20 


$ 


194,568 


Brooklyn 


$ 


■ III II I^M 


24 


$ 


169,373 


Manhattan* 


$ 




17 


$ 


229,793 


Queens 


$ 




19 


$ 


168,242 


Staten Island 


$ 




4 


$ 


174,351 


By Grade Level** 


Elementary 


$ 


13,357,513 


70 


$ 




Middle 


$ 




23 


$ 




High 


$ 


1,139,418 


6 


$ 


189,903 1 



* The high average for Manhattan includes TASC’s post-9/1 1 programming in the neighborhoods 
around Ground Zero in lower Manhattan. These projects provided services tailored to these 
students' special needs, including supplementary therapeutic care. 

** Indicates the grades served by the host school. A host school serving both elementary and 
middle grades (e.g., K-8) appears in both categories. 



In order to encourage the gradual independence of TASC-supported after- 
school projects, TASC requires individual grantees to raise matching funds from 
sources other than TASC. As part of the overall plan for implementing the TASC 
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initiative, TASC grantees were expeeted to provide 30 pereent of their total 
operating funds from non-TASC sourees during 2001-02. Eighty-three projeets 
reported on the amounts expeeted to be raised in matehing funds during that year. 
Overall, sites reported that they would raise $5,704,544 in matching funds in 
Year 4, representing 37 percent of project expenditures. 



Major Project Activities 

Objectives and intensity of after-school activities. TASC projects 
undertook activities intended to meet many objectives hut most often to help 
students huild academic skills and to expose students to positive new 
experiences. These activities were consistent with objectives reported by project 
coordinators, parents, and principals. As seen in the following table, site 
coordinators and principals were fairly congruent in their objectives, while 
parents (as reported by site coordinators) were especially likely to want the after- 
school program to provide opportunities for their children to finish their 
homework. 



Most Important Objective of After-School Projects, 2001-02, 

in Percents 



Objective 


Site 

Coordinators 

(N=74) 


Parents* 

(N=72) 


Principais 

(N=56) 


Provide academic enrichment 


41 


19 


41 


Enhance sociai or civic 
development 


26 


0 


13 


Enhance artistic development 


12 


4 


7 


Facilitate homework completion 


7 


67 


13 


Provide sports/recreation 


0 


0 


16 


Other 


15 


10 


11 



* As reported by site coordinators. 



To measure the extent to which participating students actually experienced 
particular types of activities, the evaluation measured the intensity of service in 
each major area, using an index developed for this purpose. The intensity index 
used a formula to incorporate site coordinators’ reports of the frequency and 
duration of specific activities and the proportion of students participating in the 
activity. Index values ranged from 4 at the highest level of frequency, duration, 
and coverage to 0 at the lowest level. This analytic tool permitted evaluators to 
rank activity areas by intensity, to compare average intensity levels over time, and 
to compare intensity levels across grade spans. 
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Focusing on project averages in Year 4 , three aetivity areas elustered 
together with very similar levels of intensity (2.79 to 2.71). They included artistic 
development, academic/cognitive/literaey development, and soeial/eultural 
awareness and exploration. 



Average Intensity of Activities Offered, 
According to Site Coordinators, 2001-02 (N=78) 




Average intensity level 



Projects for which comparable data were available in all three years 
remained fairly stable in intensity levels. The greatest increase appeared in the 
intensity of social and cultural awareness aetivities and the greatest deerease in 
fitness/ sports/recreation activities . 

Evaluators examined the intensity of activities intended to promote 
aeademie and cognitive development with partieular interest, given the high 
priority assigned to these activities by parents, principals, and after-school 
coordinators. Analysis of the intensity of activities in this area indicates that six 
aetivities were implemented with relatively high intensity in Year 4, ineluding 
homework help, organized reading, recreational reading, math games, word 
games, and organized writing. A review of changes in the intensity of these and 
other academic/cognitive aetivities indicates small but noteworthy inereases in 
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organized reading and tutoring, suggesting more focused efforts to improve 
participants’ academic achievement. Noteworthy declines in the period appear in 
organized writing, group instruction, computer training (likely due to projects’ 
reduced access to computers), and study skills activities. 



Intensity of Academic/Cognitive Development Activities, 
1999-2000 through 2001-02 




□ Year4(N=68) 
■ Years (N=69) 

□ Year2 (N=72) 



Average intensity ievei 



Methods for delivering program activities. An important issue addressed 
by the evaluation centered on the materials and methods used by projects to 
deliver services. To learn more about this issue, evaluators asked questions that 
sought, among other things, to determine how participants’ after-school 
experiences provided learning opportunities that differed from conventional 
school-day experiences and that sparked interest and engagement on the part of 
participants. 

A particularly popular type of program activity was designed to result 
in a final performance, such as a speech, oral report, play, or other artistic 
performance. Although these activities sometimes involved groups of students 
working together, they could be implemented as individual efforts as well. In 
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Year 4, almost all projects (97 percent) employed culminating performances, 
indicating a popularity that has remained steady over time. Overall, about two- 
thirds of projects (67 percent) said that most or all students in the after-school 
project participated in activities culminating in a performance. Eighty-seven 
percent of projects involved students in one to four such activities a year, with the 
typical annual number of such activities increasing since Year 2. 

Projects also adopted activities that were designed to result in some 
type of written product, such as a story, report, or newspaper. Like 
culminating performance activities, these were sometimes arranged as group 
efforts and sometimes as individual undertakings. In Year 4, 82 percent of site 
coordinators said that their project engaged students in activities that culminated 
in a written product. When asked about the rough proportion of students who 
participated in such activities, 68 percent of site coordinators said that some 
students participated; 14 percent said that most or all students participated. 

From the beginning of the TASC program, a favored student activity has 
been participating in extended group activities. In addition to the fact that 
students consistently expressed enjoyment of group work, site coordinators liked 
these activities because group pressure tended to encourage members of a group 
to attend regularly in order to complete their responsibilities on the project and 
not let their group members down. In Year 4, 77 percent of site coordinators said 
that students engaged in group activities that were tied in some way to one or 
more academic subject areas, that extended over a single day, and that involved 
research, writing, and discussion. This percent of projects involved in group work 
reflected an increase from Year 2, when 62 percent of TASC sites reported that 
they used group efforts. In Year 4, 24 percent of coordinators said that most or all 
of their participants had completed at least one group activity. Among this 
subgroup of coordinators, 63 percent said that most students participated in three 
to four group activities during the school year. 

Another method for engaging students in after-school activities was to 
adopt a theme that ran across and connected the project’s activities. Such 
themes might be related to a focus of the sponsoring organization (e.g., prehistoric 
life as a theme used in a project supported by a natural history museum) or a focus 
of the school, or the theme might be unique to the after-school project. In Year 4, 
76 percent of site coordinators reported that their TASC project used one or more 
themes to link activities across different program components, a percentage that 
had remained stable since Year 2. Thirty-five percent of Year 4 projects said that 
the project’s themes were coordinated with those used by the school. 

A little over a third of site coordinators (36 percent) reported in Year 
4 that they used curricula developed outside the project or school. This 
percent had increased somewhat since Year 2. Those site coordinators who 
reported using such curricula listed the following curricular programs as being 
used in their projects: 
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■ After-school Literature Project (Kidzlit) 

■ Resolving Conflicts Creatively program 

■ Putumayo “World Playground” Program 

■ Scholastic magazine 

■ Passport to Success 

■ Bronx Zoo curriculum 

■ Foundations, Inc. Literacy 

■ Radio City Club Carnivale 

■ Touchstone Discussion program 

■ American Museum of Natural History’s natural sciences 
curriculum 

■ Junior Achievement 

■ After-school Math Project (Kidzmath) 

TASC and its partners provided training to project staff on many of these 
curricular packages. 



Do Students Benefit from Participation in TASC 
Services? 

Evaluators approached this question from several directions, using 
evidence from surveys, test scores, and school attendance records. Taken 
together, these sources suggest certain types of real benefit for students. The 
discussion of student benefits first considers students’ own reports of their 
reactions to the after-school activities in which they participated, followed by 
principals’ reports of benefits to students and the school. The discussion of 
academic growth associated with TASC participation is separated into discussions 
of gains at the PreK-8 levels and at 9-12, consistent with the differing 
programming and effectiveness measures targeted at younger and older 
participants. Similarly, the discussion of changes in school attendance is 
presented separately for grades PreK-8 and 9-12. 



Student Reports of Responses to After-School Activities and 
Services 

Evaluators measured student reactions to many types of stimuli provided 
by TASC activities and services. In particular, it asked students to assess the 
degree to which the TASC project had fostered positive social interactions, built a 
sense of community within their projects, given them opportunities to grow, 
exposed them to new experiences, engaged them in learning, and given them an 
overall satisfying experience. The evaluation used separate survey forms for 
students in grades 4-5, 6-8, and 9-12. Analyses of survey responses categorized 
the mean student response to a given item as high if students rated the items in the 
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scale, when eombined aeross all the items ineluded in the scale, at 75 pereent of 
the possible maximum rating or higher (e.g., on a seale where the possible seores 
ranged from 4 to 16, a mean response of 14 or higher was classified as high). 



Students Indicating High Mean Leveis of Agreement with Scaies, 

2001-02, in Percents 



Student Scale 


Percent of 
Students with 
High Leveis of 
Agreement 


Positive Social Interactions 


Positive peer interactions (eiementary grades) 


58 


Positive staff interactions (middie and high school 
grades) 


48 


Trust of staff (elementary grades) 


59 


Trust of staff (middle and high school grades) 


44 


One-on-one adult interactions (middle grades) 


18 


Adult support (middle grades) 


24 


Sense of Community 


Program connections (all grades) 


53 


Project as a community (elementary grades) 


44 


Project as a community (middle and high school 
grades) 


30 


Youth Opportunities to Grow 


Opportunities to learn life skills (middle grades) 


56 


Opportunities for leadership (middle grades) 


3 


Sense of autonomy in the project (elementary grades) 


11 


Exposure to New Experiences 


Opportunities available through the TASC project (all 
grades) 


51 


Opportunities for mastery of performance skills 
(middle grades) 


49 


Engagement in Learning 


Academic self-esteem (all grades) 


62 


Importance of school (middle grades) 


84 


Academic benefits of TASC participation (elementary 
grades) 


55 


Academic benefits of TASC participation (middle 
grades) 


54 


Academic benefits of TASC participation (high school 
grades) 


64 



Given the high threshold used in the analysis of these seores, it is 
noteworthy that students responded as positively as they did. The significant 
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levels of educational risk represented among TASC participants make their 
positive reactions about engagement in learning especially noteworthy. 
Altogether, these patterns suggest that students may derive educational 
benefits from TASC participation. When considered in light of the efforts 
made-through the selection of program content and through staff training and 
technical assistance-to engage students in appealing, cognitively stimulating 
activities, it is reasonable to ask next if students showed evidence of academic 
learning associated with TASC participation. 



Principal Reports of Benefits of the After-School Project 

The tenor of the preceding findings is reflected in principals’ responses to 
questions about the benefits of having a TASC after-school project in their school. 

Principals perceived the primary benefits of the after-school project in terms 
of both the nonacademic and academic dimensions of the opportunities 
offered to students and families, as seen below. 



Principals’ Reports of Primary Benefits of the After-School Project, 

2001-02, in Percents (N=61) 



Primary Benefit 


Percent 


Students have opportunities to participate in 
activities not available during the regular 
school day 


95 


Parents express more positive feelings about 
the school because it provides a safe place 
for their children after school 


79 


Students receive additional opportunities to 
develop literacy skills 


66 


Teachers from the school have opportunities 
to work with children outside the classroom 


54 


Other 


5 


None 


5 



Over time, these reports of primary benefit increased on every dimension 
noted. The greatest increase was in principals’ reports of benefits in additional 
opportunities to develop literacy skills. 

In Year 4, as in previous years, principals attributed high levels of positive 
changes within the school to the after-school project. 
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Principals’ Reports of Effects of the TASC Project, 
2001-02, in Percents (N=64) 



TASC Project Effects 


Percent 
Reporting 
“Very Much” 


Enhance students’ attitudes toward school 


56 


Enhance the overall effectiveness of the school 


45 


Improve student attendance 


42 


Improve students’ safety 


41 


Enhance students’ motivation to learn 


34 


Increase parents’ attendance at school events 


31 


Contribute to improved student skills in math 


24 


Contribute to improved student skills in reading 


23 


Reduce vandalism at the school 


17 


Increase parents’ attendance at parent-teacher 
conferences 


15 



Principals’ reports of benefits “very mueh” enhaneing students’ attitudes 
toward school and improving student attendance inereased to a significant extent 
between Year 3 and Year 4. As in previous years, the vast majority of principals 
(90 percent) said that the benefits of hosting the after-school project either 
“very much” or “somewhat” outweighed the costs. 



Academic Gains Associated with TASC Participation at PreK-8 
Levels 

To assess aeademie ehange assoeiated with TASC partieipation, 
evaluators ereated longitudinal data files for every student partieipant and 
nonpartieipant in the TASC evaluation sample who met minimal eriteria 
regarding data availability. Evaluators then eomputed changes in performanee on 
the city and state English Eanguage Arts/reading (EEA) and math achievement 
tests for eaeh TASC partieipant by subtracting the standardized seale seore in the 
year prior to the student’s first enrollment in a TASC after-sehool projeet from his 
or her standardized seale seore after participation in the program.^ That is, for an 
estimate of the ehanges in student performanee after one year of exposure to 
TASC programming, evaluators eomputed the differenee in the proportion of 



^ Information on the standardization of scale scores is available in a technical paper. 
Standardization of scale scores is necessary to permit individual-level analyses across years and 
across grade levels. With standardization, differences in scale scores across grade levels represent 
changes in student performance that are independent of the differences in the particular scale 
applied to scores at each grade level. 
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possible scale-score points that a student earned in one grade level compared to 
the proportion of possible points earned at the next. Evaluators repeated the 
procedure to assess the change in student performance after participation in the 
TASC program for two years, computing the difference in the standardized scale 
score in the year prior to exposure to the TASC program and the standardized 
score after two years of exposure, two grade levels later.^ 

For nonparticipants, evaluators computed change in performance on the 
FLA and math achievement tests by subtracting the nonparticipant’s standardized 
scale score in the year prior to enrollment in a school hosting a TASC project 
from his or her standardized score after a year of nonparticipation. 

Evaluators took steps to make the comparison of participants and 
nonparticipants as fair as possible by first considering the differences in factors 
associated with students’ test performance across participants and nonparticipants. 
Factors included in this analysis included family income, gender, race, and 
eligibility for specialized educational services, such as special education and 
classes for English Fanguage Feamers. Evaluators created a statistical model, 
described in a separate technical paper, to estimate the change in performance on 
the EFA and math achievement tests that would have been expected for each type 
of participant, had they not participated in the program. The comparison between 
this predicted change and each participant’s actual change in performance then 
represents the study’s estimate of the effect of the TASC program. 

This approach to analyzing achievement change associated with 
participation in TASC-supported projects corrects for self-selection bias 
represented in the student characteristics included in the regression equation and 
for differences in performance during the year prior to first enrolling in a TASC 
project. Evaluators also considered how to take into account differences in 
unmeasured characteristics of students and their families that may be associated 
with different levels of student achievement. Examples of possible differences 
include the student’s motivation to do well in school or the importance the family 
places on success in school. In developing the study’s approach, evaluators 
assumed that the differences on the unmeasured characteristics associated with 
differing performance on achievement tests or frequency of school attendance 
were controlled for by including the prior-year test scores in the equation for 
predicting expected gains on test scores and also by including the prior-year 
school attendance rate in the estimate of gains in school attendance. 

Aggregate test score gains. On average, participants made greater 
gains in math than did similar nonparticipants. Students who participated in 
TASC the most consistently and for the longest period of time experienced 



^ Only participants and nonparticipants who had a normal grade progression (i.e., were not left 
back and did not skip grades) and for whom the evaluator had a test score in the year prior to 
exposure to TASC and in the target year (i.e., after one and/or two years of exposure to the TASC 
program) were included in this analysis. 
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the greatest gains. In particular, students participating actively in TASC projects 
for more than a year showed significantly greater gains in math than did their 
nonparticipating classmates, and students participating for two or more years 
showed even more substantial improvements in performance than 
nonparticipating students. In order to compare the magnitude of the change in 
performance of participants compared to similar nonparticipants, the evaluation 
computed the effect size, as the difference between the true change in 
participants’ performance and their expected change, dividing the difference by 
the standard deviation of the nonparticipants’ predicted change in performance. 
Evaluators considered an effect size of 0.10 or larger to be substantive. 

Among all TASC participants, the average change in scores on the 
mathematics test was 1 .4 standardized scale-score points more after two years of 
participation than would be predicted from students’ characteristics. This 
indicates that students gained an amount equivalent to 1 .4 percent more of the full 
range of possible standardized scale-score points in math after two years than did 
similar nonparticipants. The difference between this change in performance and 
that of nonparticipants was statistically significant with an effect size of 0.42, 
indicating that participation in TASC was associated with a substantial change in 
performance. Participants earned 0.2 percent more of the possible points than 
similar nonparticipants after one year of TASC participation, which was 
statistically significant but not substantively different from nonparticipants’ 
change in performance. 



Difference in Change in Mathematics Standardized Scaie Scores 
for Aii Participants, Grades 3-8 



Years of Participation 


Difference from 
Expected Change 
in Performance 


Expected 
Change in 
Performance 


Effect Size 


N 


1 Year 


0.2* 


-2.0 


0.06 


11,409 


2 Years 


1.4* 


-3.9 


0.42 


2,666 



* Indicates significance at p<.05 level. 

Effect sizes greater than 0.10 are In held type. 



After one year of exposure to TASC, students who were active 
participants in TASC after-school projects (attending at least 60 percent of the 
possible days while enrolled and attending at least 60 days during the school year) 
in every year they participated gained more in math performance than expected 
based on their characteristics. Students who were active TASC participants also 
experienced gains in math scores that were significantly larger than expected after 
two years of exposure to TASC. The effect size characterizing this two-year 
difference was approximately 0.79, which indicates a substantive difference. 

After one year of participation, active participants gained 0.5 standardized 
scale-score points more than would be predicted from students’ characteristics. 
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This indicates that students gained an amount equivalent to 0.5 percent of the full 
range of possible seale-seore points in math after one year than did similar 
nonpartieipants over the same period. After two years of partieipation, active 
partieipants earned 2.6 standardized seale-seore points than did similar 
nonpartieipants. 



Difference in Change in Mathematics Standardized Scaie Scores 
for Active Participants, Grades 3-8 



Years of 
Participation 


Difference from 
Expected Change 
in Performance 


Expected 
Change in 
Performance 


Effect Size 


N 


1 Year 


0.5* 


-2.0 


0.13 


5,543 


2 Years 


2.6* 


-3.8 


0.79 


1,148 



* Indicates significance at p<.05 level. 

Effect sizes greater than 0.10 are In held type. 



No corresponding achievement differences were evident in 
ELA/reading tests. The reason for the disparity in reading and math effeets may 
he in one or both of the following explanations; 

■ The systematie nature of math learning, in whieh eaeh skill builds 
on the preeeding skill, may benefit from opportunities for students 
to eomplete their homework every day and to get help with it. By 
eontrast, literaey-related learning builds on more eomplex skills 
and experienees, and henee may be less elosely linked to 
homework eompletion. 

■ Literaey-related learning may depend on the student’s immersion 
in a literaey-rieh environment from an early age, as further 
augmented through instruction by a trained reading speeialist. As 
a result, literacy skills may not be susceptible to immediate 
improvement through the informal enriehment aetivities that, along 
with homework help, make up the eontent of most TASC after- 
sehool aetivities. 

Gains for subgroups. In addition to examining aggregate aehievement 
ehange, evaluators also eonsidered whether all TASC partieipants experieneed 
similar benefits from project participation. The evaluation addressed this question 
by repeating its analyses of test-seore changes for varied eategories of students. 

As in the overall analysis, the eomparisons examine the ehanges in standardized 
test seores expeeted for partieipants if they had not been part of the TASC after- 
sehool program, eompared to the ehanges in test seores aetually observed for 
different partieipant groups. Beeause of the eonsistent evidenee of overall benefit 
in mathematies, reporting on the subgroup analyses is eonfined to math. The 
analysis examined results for subsets of partieipants who met the active 
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participation threshold and who possessed known charaeteristics in eaeh of the 
following areas: prior performanee on the eitywide assessments, free luneh 
eligibility, raee/ethnicity, speeial education status, English Language Learner 
status, reeent immigrant status, and gender. For most of the analyses, greater 
math gains were found among students whose personal characteristics 
indicated greater educational risk. 

Participation in the TASC after-sehool program provided additional 
benefits to students who initially seored at or above grade level in math, and also 
benefited students who initially seored below grade level and partieipated aetively 
in TASC for at least two years. 

■ Students who scored at or above grade level in math 
(proficiency levels 3 and 4) in the year prior to enrolling in a 
TASC projeet had signifieantly larger gains than predieted after 
both one and two years of active partieipation. 

■ Students who scored helow grade level in math (proficiency 
levels 1 and 2) in the year prior to enrolling in a TASC project had 
mixed results in math. They gained signifieantly less than 
predieted after one year of aetive partieipation but signifieantly 
more than expeeted after two years of partieipation. 

Students from families who were at or below the poverty level also 
showed larger than expected gains on the math exams after partieipation in a 
TASC projeet. 



■ Participants who were eligible for free lunch in the year prior to 
TASC partieipation gained more points than expeeted in math after 
both one and two years of participation. 

■ Participants who were not eligible for free lunch in the year 
prior to TASC partieipation gained fewer points than expeeted 
after partieipating in TASC for one year but more points than 
expected after two years. 
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Number of Standardized Scale-Score Points 
Gained in Math Above Similar Nonparticipants 
by Active Participants, Grades 3-8, by Proficiency Level 




* Indicates a test score gain that is significantly different (p<.05) from similar nonparticipants 
Effect sizes greater than 0.10 are In bold type. 



Ns for Each Analysis Group 


After 1 Year 


After 2 Years 


Proficiency Levels 1 and 2 


3,535 


703 


Proficiency Levels 3 and 4 


2,008 


445 



Participants’ outcomes varied by racial group, with black and Hispanic 
participants showing greater gains over similar nonparticipants than did white and 
Asian participants. (Findings for different groups cover varying time periods, 
depending on sample sizes.) 

■ Black students who regularly participated in TASC gained more 
points than expected in math after one and two years of exposure 
to TASC (with effect sizes of 0.10 and 0.58, respeetively). 
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Number of Standardized Scale-Score Points 
Gained in Math Above Similar Nonparticipants 
by Active Participants, Grades 3-8, by Free-Lunch Eligibility 




* Indicates a test score gain that is significantly different (p<.05) from similar nonparticipants. 
Effect sizes greater than 0.10 are In bold type. 



Ns for Each Analysis Group 


After 1 Year 


After 2 Years 


Free Lunch Eligible 


4,592 


979 


Not Free Lunch Eligible 


951 


169 



■ Hispanic students who participated regularly gained more points 
than expected after one and two years of exposure to TASC (with 
effect sizes of 0.19 and 0.95, respectively). 

■ White and Asian students gained only the expected number of 
points in math after one year of exposure. 

Participants who qualified for certain special-needs categories in the year 
prior to attending a TASC project and who attended TASC regularly performed 
better than similar nonparticipants, although participants in a third category did 
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not. (Changes are only reported for a single year of participation because of small 
sample sizes in the second-year data for these subgroups.) 

■ Special education students who attended TASC projects gained 
more points than expected on math tests after a year of 
participating in TASC (effect size of 0.24). 

■ English Language Learners gained more points than expected in 
math after a year of TASC participation (effect size of 0.11). 

■ Recent immigrants, in contrast, gained fewer points than expected 
in math after a year of TASC participation (negative effect size of 
0 . 20 ). 

Male and female participants’ outcomes were similar to those of all 
participants who attended the program regularly. Both boys and girls who 
attended TASC regularly gained more points than expected on the math tests after 
one and two years of exposure to TASC. 



Academic Gains Associated with TASC Participation at Grades 9-12 

The analysis of academic gains at the high school level is more 
problematic than at the earlier grades because of the following conditions: 

■ The sizable pre-existing differences between TASC participants 
and nonparticipants in the high school grades, with TASC 
participants displaying fewer indicators of educational risk before 
enrolling in TASC projects than nonparticipants 

■ The absence of comparable annual measures, which would make it 
possible to assess change from one year to the next 

This difference, among other things, precluded the use of the 
regression-based controls used to equalize the pre-existing 
characteristics of TASC participants and nonparticipants in the 
earlier grades. 

■ The presence of only six high school projects in the evaluation 
sample, each with very different after-school enrollment policies 
and TASC activities 

Project-level variations, including the differing availability of 
student data across sites and differing after-school enrollment 
policies, persuaded evaluators to examine student academic growth 
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on a project-by-project basis, rather than as summed and averaged 
across all projects. 



These issues raise questions about the interpretation of the overall finding 
regarding comparative academic gains for high school participants. This finding 
is that TASC participants in projects serving the high school grades passed 
more Regents exams and earned more credits toward high school graduation 
than did nonparticipants in the same schools. The finding and the caveats that 
surround it are discussed in this section. 

Performance on Regents exams. Evaluators examined participants’ and 
nonparticipants’ performance on Regents exams using several types of 
comparisons, including percent of students passing each major exam and the 
grade level at which students passed the exam. The culminating analysis 
examined the percent of students passing five or more exams by grade level. This 
analysis demonstrates that, in the four sites for which fairly complete Regents 
data are available, participants passed more exams earlier in their high school 
career than did nonparticipants, with active participants (i.e., those high school 
students participating in the TASC project 20 percent or more of the available 
days and at least 20 days during the school year) especially likely to pass five 
exams by the end of twelfth grade in two of the four schools. 

Credits earned toward high school graduation. In the sites for which 
sufficient data on high school credits earned were available, the average number 
of credits earned among participants was significantly higher than the average for 
nonparticipants. Again, this pattern is not surprising given the higher incidence of 
educational risk factors among nonparticipants. 

To hold one of the major risk factors constant across participants and 
nonparticipants, evaluators analyzed credits earned by the end of ninth grade in 
light of students’ prior achievement levels. In this analysis, evaluators 
categorized participants and nonparticipants by their eighth-grade proficiency 
levels. In the two projects for which this analysis was possible, participants 
across seven out of ten cluster comparisons had earned more credits at the 
end of ninth grade than had nonparticipants with the same eighth-grade 
proficiency level in the subject. Because of the controls for prior performance 
included in this analysis, the pattern revealed here lends weight to a conclusion of 
benefits accruing to high school TASC participants. 



Attendance Changes Associated with TASC Participation 

Two characteristics of school attendance rates create challenges in 
analyzing the relationship between program participation and changes in school 
attendance. First, the relatively high rate of attendance citywide establishes a 
ceiling on attendance-rate improvement, since for most students there is little 



36 




opportunity for substantial increases in school attendance to oecur. Second, the 
varying pattern in attendance rates aeross grade levels, combined with the 
differences in the distribution of participants and nonparticipants across grade 
levels, requires adjustments to the raw attendance data through weighting. The 
weights construeted in the evaluation control for the differences in participants’ 
and nonparticipants’ distribution across grade levels. All of the findings 
presented for school attendance were computed using weighted data. 

Change in attendance patterns at grades PreK-8. On average, 
participants attended sehool more frequently than nonparticipants in the year prior 
to enrolling in TASC. The average weighted attendance rate for nonparticipants 
in grades PreK-8 was 91.73, while for partieipants it was 92.88 pereent. Students 
who were aetive participants in their first year in TASC serviees attended school 
93.47 percent of the time in the year before they enrolled in the projeet. 

To estimate the relationship between participation in a TASC project and 
students’ sehool attendanee, analyses foeused on determining whether the gap 
between participants and nonparticipants increased over time. Evaluators found 
that the gap between the attendance rates of TASC partieipants and corresponding 
nonparticipants grew slightly after one year of participation in a TASC after- 
school project. After one year of TASC exposure, the average attendance rate 
among all participants increased by 0.53 percentage points, compared with 
an increase of 0.11 percentage points for nonparticipants, for a net difference 
of three-quarters of a day over a 181-day school year. The corresponding 
increase among aetive partieipants was 0.75 percentage points, yielding a net 
difference in gains in school attendance of 1 .2 days over the sehool year. 



School Attendance Rates and Change in Rates 
After One Year of Exposure to TASC, PreK-8, Weighted 



Student Group 


School Attendance Rate 


Average 
Individual 
Percentage Point 
Change 


Statistical Comparison to 
Nonparticipants 


Average In 
Base Year 


Average 
After 1 Year 


Significant at 
p<.05 


Effect Size 


Active 

Participants 

N=10,895 


93.47% 


94.22% 


0.75 


Yes 


0.09 


Aii Participants 
N=32,941 


92.88% 


93.41% 


0.53 


Yes 


0.05 


Nonparticipants 

N=44,827 


91.73% 


91.84% 


0.11 


- 


- 



Because attendance rates vary by grade and because participants and nonparticipants are 
distributed differentiy across grades, the evaiuation weighted the schooi-attendance data to 
equaiize the proportion of participants and nonparticipants enroiied at each grade. 
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The gap in school attendance between the students who participated in a 
TASC after-school project for two years and corresponding nonparticipants also 
grew significantly. After two years of participation, the school attendance 
rates for all TASC participants increased hy 0.68 percentage points, 
compared with 0.38 percentage points for nonparticipants, the equivalent of 
attending an additional half-day of school per year, compared to 
nonparticipants. Over the same period, the school attendance rates of active 
participants increased by 0.80 percentage points, for an additional three-quarters 
of a school day per year, compared to nonparticipants. 



School Attendance Rates and Change in Rates 
After Two Years of Exposure to TASC, PreK-8, Weighted 



Student Group 


School Attendance 


Average Individual 
Percentage Point Change 


Statistical Com 
Nonpartici 


parison to 
lants 


Average in 
Base Year 


Average 
After 2 Years 


Base to 
First Year 


Base to 
Second Year 


2 Year Change 
Significant at 
p<.05 


Effect Size 


Active Participants 
N=5,965 


94.08% 


94.83% 


.92 


.80 


Yes 


0.06 




93.59% 


94.27% 


.95 


.68 


Yes 


0.04 


Nonparticipants 

N=20,605 


92.46% 


92.85% 


.68 


.38 


-- 


-- 



Because attendance rates vary by grade and because participants and nonparticipants are 
distributed differentiy across grades, the evaiuation weighted the schooi-attendance data to 
equaiize the proportion of participants and nonparticipants enroiied at each grade. 



The attendance advantage associated with active TASC participation 
through the middle grades is particularly noteworthy. Although the 
attendance rates of nonparticipants in the host schools consistently declined 
between the fifth and eighth grades, this pattern did not characterize the school 
attendance of either TASC participants generally or active TASC participants. 
After one year of TASC participation, school attendance rates of all participants 
and of active participants declined between fifth and sixth and between sixth and 
seventh grades, but increased between seventh and eighth grades. For each of 
grades 5-8, the difference between participants and nonparticipants was 
positive and significant, and the effect sizes were greater than 0.10. For 
example, the difference in the attendance gain of participants and nonparticipants 
measured from seventh to eighth grade was 1.5 percentage points or the 
equivalent of a net gain of 2.7 school days in a 181-day school year. 
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Change in School Attendance from the Year Before Enrollment 
to the First Year After Exposure to TASC, 
by Grade, PreK-8, Weighted 




Effect sizes are noted in parentheses. Effect sizes greater than 0.1 0 are in boid type. 



After two years of partieipation in TASC, corresponding nonparticipants 
experienced decreases in school attendance between fourth and sixth grades, fifth 
and seventh grades, and sixth and eighth grades. Among TASC participants, 
school attendance rates increased between grades 4 and 6, and decreased less than 
among nonparticipants between grades 5 and 7 and grades 6 and 8. 
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Change in School Attendance from the Year Before Enrollment 
to the Second Year After Exposure to TASC, 
by Grade, PreK-8, Weighted 



8 




-4 



Effect Size 



Active Participants (0.09) 


(-0.09) 


(-0.02) 


(0.01) 


( 0 . 10 ) 


( 0 . 10 ) 


( 0 . 36 ) 


( 0 . 38 ) 


All Participants ( 0 . 10 ) 


(- 0 . 10 ) 


(-0.01) 


(-0.01) 


(0.06) 


( 0 . 13 ) 


( 0 . 16 ) 


( 0 . 15 ) 


PreK to 1 ' 


Kto2* 


1 to3‘ 


2 to 4* 


3 to 5 * 


4 to 6* 


5 to 7* 


6 to 8 



I ■Nonparticipants N=20,603UA\\ Participants A/=//,259D Active Participants N=5,965\ 

* Indicates an attendance rate change that is significantiy different (p>.05) from simiiar 
nonparticipants. Effect sizes are noted in parentheses. Effect sizes greater than 0.1 0 are in boid 
type. 



The evaluation also examined students’ change in attendance rates in light 
of their school attendance rates prior to TASC enrollment, on the theory that, all 
things being equal, students were most likely to continue to attend school at the 
same rate that they had attended before. To facilitate this analysis, evaluators 
categorized students’ attendance rates in the year prior to their TASC 
participation into quartiles. This method permitted evaluators to compare the 
attendance rate changes of participants with those nonparticipants who displayed 
the same attendance history. This analysis, which controlled for students’ 
prior level of school attendance, indicated attendance improvement for 
TASC participants at every quartile. 

Change in attendance patterns at grades 9-12. The evaluation also 
examined students’ high school attendance rates. Although TASC participants 
attended at consistently higher rates than nonparticipants, this pattern was not 
surprising, given TASC high-school participants’ lower incidence of risk factors, 
compared to nonparticipants. Because of this difference, evaluators analyzed 
change in high-school participants’ attendance in light of their attendance rates in 
the year prior to their TASC participation, using the method described above. 
Although analyses that organized students by prior attendance quartile 
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found generally decreasing high-school attendance at each quartile, the 
attendance-rate declines were smaller or even reversed among participants 
hut not nonparticipants. 



Summary of Patterns in Educational Performance Associated with 
TASC Participation 

Viewed from the most eautious perspeetive, it is safe to say that 
involvement in TASC-supported after-sehool aetivities and experiences supported 
participants in improving their educational performance relative to 
nonparticipants. This finding is evident across grade levels and in both test score 
performance and school attendance. Whether these differences were driven by 
after-school participation itself or by measured or unmeasured differences 
between participants and nonparticipants is a harder question to answer. 

However, considering information on all the program and participant factors 
available to the evaluation and the steps taken to account for pre-existing 
differences across participants and nonparticipants, the evaluation team concludes 
that, at the PreK-8 level, the comhination of (1) the similarity of participants 
and nonparticipants on all measured demographic and prior-performance 
variables, especially following statistical adjustment, and (2) the sheer size of 
the mathematics differences suggests the likelihood of real after-school 
benefits for participants, especially for those who participated the most 
regularly and for more than a year. Similarly, the attendance gains, 
especially for participants in the middle grades, are also noteworthy. At the 
9-12 level, the team concludes that after-school involvement supported 
participants in making greater gains than they otherwise would have made 
especially in school attendance. Although the suggestion of academic gains at 
the high school level is intriguing, problems in measurement, sample size, and 
participant/nonparticipant comparability preclude conclusions in this area. 

Even taking the worst case regarding selection bias, however, students 
who enrolled in and attended TASC-supported after-school projects enjoyed the 
experience enough to attend on a regular, sustained basis. They also used their 
after-school experience as an opportunity to build skills and personal assets that 
engaged them more fully with their schooling and improved their educational 
performance. 
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What Practices Are Associated with the Greatest 
Academic Benefits for Students? 

An important question for the evaluation eoneemed the extent to whieh 
specific project-level characteristics may have exerted particular influence on 
participant outcomes. Answers to this question are important because they can 
help TASC determine whether to modify the program model, to emphasize or 
deemphasize particular program components, or to monitor certain elements of 
project implementation with special care. They can also help sponsoring 
organizations and project site coordinators to make decisions about project 
characteristics that may shape after-school effectiveness. 

Evaluators approached this question using the program theory of change 
as their central tool. Using an approach that the evaluators termed “backward- 
mapping,” they looked for critical project- level characteristics by tracing 
backward through the change theory to identify program components that figured 
prominently in the hypothesized causal chain. With certain characteristics 
identified as being possibly influential in shaping participant outcomes, evaluators 
examined a series of relationships between specific project activities, program 
practices and relationships, staff qualifications, professional development 
opportunities, and other factors to determine the ways in which project design and 
implementation affected participant outcomes. On the outcome side of this 
analysis, evaluators used test data from nonparticipants to build a model that 
predicted change in test scores based on student characteristics. They then ran 
each participant through the model to compute both their actual and predicted 
test-score changes. Next, they compared predicted test score changes to actual 
changes. Finally, they conducted tests to determine whether the average of the 
differences between participants’ actual and predicted test-score changes were 
significantly different and to compute the effect size. Because of the absence of 
test-score comparisons at PreK-2 levels, the limitations of the educational 
performance measures at the high school grades, and the small Ns for TASC 
participants in grades 9-12, the analyses were confined to grades 3-8. 

Another limitation of this analysis is the narrow band of variation across 
projects on many important variables of interest. This project-level similarity is 
due in part to the specificity of the TASC after-school model, which requires, for 
example, a full-time site coordinator and an expectation of regular after-school 
attendance in projects serving grades PreK-8. The similarity is also driven in part 
by the generally uniform pressures on host schools, all of which experience 
significant external and internal pressure to improve student academic 
performance. A third factor promoting project similarity is TASC training and 
technical assistance, which has encouraged certain types of program activities that 
projects and program monitors have found to be effective in stimulating student, 
parent, and host school interest and support; examples of such activities include 
culminating performances, group activities, and the use of regular-day teachers as 
educational advisors to less experienced after-school staff. However, despite 
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similarities across project activities and staff configurations, evaluators found 
certain project characteristics that both varied across projects and also were 
associated with positive participant outcomes in both subject areas. 

An especially important set of associations, based on participant outcomes 
compared across projects, focused on the activities that projects selected as their 
focus, as assessed using the evaluation’s measure of activity intensity. Findings 
regarding project activities are: 

■ Participants made more positive one-year gains in both ELA and 
math test scores in TASC projects that offered a high intensity 
of academic and cognitive development activities (effect size of 
0.26 in math and 0.15 in ELA). Information collected through 
surveys and site visits makes it clear that projects achieved high 
intensity in academic and cognitive activities without necessarily 
making after-school an extension of the school day. Instead, 
projects used extended, multidisciplinary activities, often involving 
groups of students working together and also often culminating in 
performances or major products or publications. 

■ Participants also made more positive one-year gains in TASC 
projects that offered high intensity in activities focusing on 
fitness, sports, and recreation (effect size of 0.14 in math and 
0.20 in ELA). Evaluators speculate that these activities influenced 
participants both by drawing them into the after-school program 
and promoting high attendance and also by providing the physical 
exercise needed for subsequent mental acuity. 

Certain staffing and supervisory practices were also associated with 
participant learning. These practices included the following: 

■ In TASC projects in which the site coordinator had a teaching 
certificate, participants performed better than participants in other 
sites (effect size of 0.13 in math and 0.26 in ELA). This difference 
may represent two types of influences exerted by site coordinators 
with teaching qualifications: (1) their relatively greater ability to 
promote high-quality after-school teaching through coaching and 
demonstration and (2) the likelihood that they established positive 
professional relationships with the host school and hence that after- 
school learning activities were well coordinated with participants’ 
learning experiences during the regular school day. 

■ In TASC projects where the site coordinator required project 
staff to submit lesson or activity plans, participants made greater 
gains on tests than in programs where the site coordinator did not 
require lesson plans (effect size of 0.14 in math and 0.17 in ELA). 
Evaluators speculate that the preparation and review of written 
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plans occurred mainly in projects in which student learning was a 
high priority. 

■ In sites where at least 25 percent of project staff had a four- 
year college degree, participants had more positive changes in test 
scores than in TASC sites with a lower proportion of sehool staff 
members with such degrees (effect size of 0.14 in math and 0.13 in 
reading). Staff with college degrees may be better able to see and 
to exploit the varied learning opportunities embedded within 
themes and topics adopted by after-sehool projects. 

Positive relationships with one-year student gains in reading and math 
were also found in three additional types of staff eharacteristics. These included 
the employment of staff who: worked in the host sehool in some capaeity (with 
associations found when more than 25 pereent of staff reported this 
eharacteristic), spoke a language in addition to English (when more than 50 
pereent of staff reported this characteristic), and were under the age of 35 (when 
more than 75 percent of staff reported this characteristic). These assoeiations 
underscore the importanee of staff capacities and staff interaetions with 
partieipants in shaping student learning outcomes. 

The preceding findings must be understood in light of the qualifieations 
noted at the beginning of this section: Because all TASC projects shared 
certain key characteristics, the backward-mapping analysis could not assess 
the impact of those program features on student outcomes. Qualitative 
evidence suggests that certain features were espeeially important, including: 

■ The partnership between nonprofit sponsoring organizations and 
host sehools 

■ The location of after-school services within participants’ own 
schools 

■ The employment of a full-time project site coordinator 

■ The expectation that participants would attend the after-sehool 
project almost every day 

Future evaluation of the TASC program model should be designed to permit 
assessment of the student impact of these project features. 
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What Do These Evaluation Findings Imply for 
TASC? 

Taken altogether, the evaluation findings reported here indieate that 
TASC is indeed implementing its enriched model of after-school services and 
that these services are meeting with success in at least three erueial areas. First, 
the nonprofit sponsoring organizations and host sehools have proved fully eapable 
of administering the TASC model essentially as intended. Second, the projects 
are meeting a market test of popular appeal, as evidenced by their proven capacity 
to attract and retain their targeted customers. Third, the projects are supporting 
improvements in educational performance by participants, especially those who 
attend a TASC project on a regular basis and for more than a year. Looking 
forward, TASC’s challenge will be to consolidate and extend these successes in a 
tight fiscal environment. 

Project sponsors and host schools are administering the TASC model 
as intended. Although sponsors and project staff adapt their TASC projects to 
meet the needs of their students and their host schools, projects are being 
administered in conformity with the intended program model. In particular, 
projects offer varied academic and nonacademic services to students, in order to 
engage them in positive learning experiences that differ from the regular school 
day but complement it. Even in an environment of tight budgets and the need to 
secure funds from sources in addition to TASC, projects are staffed with people 
who are well qualified for their duties and who take positive advantage of the 
professional development opportunities offered to them. 

Projects are attracting and retaining participants. Despite the 
attractions of urban life outside project walls, TASC projects are sufficiently 
appealing and effective in reaching out to students, parents, and educators such 
that they attract and retain large numbers of students. In general, these after- 
school participants display many characteristics of educational risk and hence 
appear likely to benefit from the types of enriched learning opportunities that 
TASC projects provide. 

Projects are supporting improvements in participants’ educational 
performance. With the conditions and caveats described in this report, TASC 
projects are supporting meaningful improvements in participants’ educational 
performance, especially in math achievement and school attendance. Despite 
projects’ extensive opportunities for students to use reading skills, existing 
evidence does not point to student growth in literacy that can be linked to after- 
school participation. 

TASC’s challenge is to consolidate and extend these successes in a 
tight fiscal environment. TASC’s overall strategy for sustainability has been to 
form partnerships with institutions and funding sources that value TASC’s focus 
on high quality and service expansion. While that strategy has resulted in fairly 
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rapid growth in the number of projeets and the number of students served, TASC 
may find itself inereasingly ehallenged to retain the more eostly elements of its 
enriehed model (e.g., full-time site eoordinator, varied aetivities that respond to 
student interests, extensive opportunities for staff training) as it also reduees the 
proportion of projeet eosts that it eovers. Findings of this evaluation suggest that, 
in making the inevitable tradeoffs that this dilemma will pose, TASC will do well 
to lean toward retaining its speeial foeus on quality even if this strategy requires 
cutbaeks in numbers. TASC’s quality foeus makes it a national leader as a 
provider of high-quality after-sehool serviees, and this role may be the only 
reliable route to long-term program sueeess. 

A step in eonsolidating its sueeesses will be for TASC to examine why its 
projeets are not experieneing positive growth in students’ literaey skills and then 
to adopt new polieies and supports that ean promote sueh growth. A possible 
initial inquiry to support this examination will be to identify those projeets where 
partieipants’ literaey growth outpaees that of nonpartieipants. Analysts ean then 
look in-depth at these projeets to learn more about how these projeets differ from 
other projeets-whom they hire and how staff are supervised and trained, what 
aetivities staff undertake with students, how students are matehed up with 
aetivities, and how after-sehool learning experienees relate to the literaey 
instruetion provided in the regular sehool day. Findings of this work ean then be 
shared with other TASC projeets to support their improvement efforts in the 
erueial area of literaey learning. 



* 

More information on the following topies is available in supplementary 
papers that report additional information and findings from the evaluation: 

1 . Overview of the TASC program, ineluding the program model and 
speeial initiatives launehed to inerease program quality, reaeh, and 
sustainability 

2. Charaeteristies of sehools and students partieipating in the TASC 
program 

3. TASC aetivities and students’ reaetion to them 

4. Analysis methods and findings regarding ehange in partieipants’ 
edueational performanee and sehool attendanee 



46 




